\chapter{Related research}\label{chap:3}
%THE SYSTEM ARCHITECTURE
%
%
This chapter presents the recent approach for automatically classifying malware into malware families. For the reason that the detection based on static signature is no longer effective in chapter \ref{chap:2}, new malware classification method is required to detect malware using avoidance technique.

\section{Flow graph}
Another approach is using emulator to automatically unpack the packing malware, then from reverse code produce flowgraph, flowgraph matching to perform classification\cite{silvio}. The system is created by the approach as the followings:

\begin{itemize}
\item Fist, System automatically unpacks malware based on application level emulation, and then uses entropy analysis to detect.
\item Second, Produce control followgraph by using graph invaiant based on signature in order to measure similarity between malware.
\item Next, Genetic string based on control flow signature in order to be able to use string edit distancce. Automatically unpack malware based on application level emulation.
\item Finally, Malware classification uses a set of  similarity functions and a set of similarity search algorithms to identify benign and malicious classes.
\end{itemize}

The disadvantage in flowgraph approach is high cost of runtime complexity. Firstly, runtime complexity of malware classification is $O(N\log{M})$ where M is the number of control follow graphs in the database, and N is the number of control follow graph input binary \cite{silvio}. In addition, to identify two flowgraphs is $N^{3}$, considering N is the number of nodes in each graph. Further more, needs to unpack the sample if it was packed with an executable packer. Therefore, this approach is ineffective in malware classification system with a large number of instances.

In addition, malware classification based on flowgraph approach cannot accurately detect metamorphic malware. As presented in chapter \ref{chap:2}, metamorphic malware can recode itself with program follow modification, function reordering. As a result, it is hard to classify malware based a set  similarity between follow graph.
\section{Optimizing decision tree in malware classification system using Genetic Algorithm}
Malware classification system can use machine learning classifier. The main idea of machine learning technique classifier is a search algorithm that is learnt from externally supplied instances to produce a concise model of the distribution, which then  makes prediction about new  instances. Current machine learning classifier in malware classification include: Naive Bayse, Suport Vector machine, Decision tree, K-nearest Neighbor \cite{mohd}. This approach uses combining Genetic algorithms with Decision tree. The data set is separated in training data set and testing data set. The data set includes:
\begin{verbatim}

// Malware classes for chromosome
hash_map<MalwareClass*,int> _Dataclasses;
hash_map<MalwareClass*,int> _Appsclasses;
hash_map<MalwareClass*,int> _Sysclasses;
hash_map<MalwareClass*,int> _Dosclasses;

\end{verbatim}
Genetic algorithm is a method analogous to the process of natural evolution. Genetic algorithm is used to optimize decision tree for accurately classifying malware. Malware classification system, which is implemented by the combining generic algorithm with decision tree algorithm approach, is to classify malware into two classes: benign program and malicious program, not to detect semantic characterization of malware by classifying malware into families. Therefore, the combining generic algorithm with decision tree algorithm approach cannot be used to detect semantic characterization of malware.
\section{Conclusion}
To achieve fast malware classification, two approaches described as above can not be used in the system. New approach for fast malware classification system will be introduced in the next chapter. 